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M231t "An>ly8i»" 



Intr jduction 

Thi. •tudy Btrictly lie. »oinewh*t out.ide the iii«in evaluation project, in 
that in the 17 evaluation studie. "proper" the Research Group was re.pon.ible 
for the formulation of genr.ral evaluation criteria apainat which the individual 
pn>gran»e. were examined in detail. With M231, however, it wax the Cour.e Team 
which approached the Re.earch Group for aa.i.tance in evaluating a.pect. of the 
course for two apecific purpoaea: 

(a) in order to make adju.tment. to the content of the M231 courae in 1975 
(the second year of ita life) 

(b) in order to take informed decision, on the presenttition of material for 
M332 "Complex Analysis" (which w*s being prepared by the same course team 
for presentation in 1975). 

The two major ar«as of preoccupation for the M231 course team were: 

(1) the amount of work included in the course as a whole, and its organiMtion 

(2) the technique, used in some of the television programmes. 

Since the planned third-level cour.e was intended to be presented in roughly the 
Mme way, with a .ikilar organisation of work material, and poa.ibly «>me of the 
«uiie progranme technique., our brief was to discover what problem, .tudent. were 
having in using the M231 material.. 

The M231 evaluation, then, wa. very different in orientation from that of 
the other .tudie. carried out in 1974? Firstly, it was aimed at finding answer, 
to specific queation. rai.ed by the course team. Secondly, it attempted a more 
global examination of the course a. a whole - including the written units - than 
would have been appropriate to the other more detailed evaluation studies. 
However, many of the findings from the M231 study contain implication, for course 
design in general, and indeed pn>vided a number of insight, for the Research Group 
as to potentially fniitfUl areas for investigation in the later, more detailed 
studies. For these reasons, it is presented as the first broadcast evaluation 
report of the 1974 series. 

The Problems 

An original intention of the M23I course team was that certain sections of 
the. unit, could be made optional. In fact students had been warned that changes 
would be made to the status of some sections and that these would be identified 
in the "Stop Press". Consequently, it would be possible to increase or decrease 
the amount of content in the course to some extent. Accordingly we set out to 

(1) determine a 'normal' time for the period of study for a 
particular unit, 

(ii) identify sections in the unit which were abnormally difficult, 
(iii) find the 'normal' time of study for those sections of the unit 
which could be made optional. 



U See the General Introduction to any of these reports for a descripti 
aim., criteria and method, u.ed. 
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Infomuitlon on these aspects was intended to make it possible for the course 
team to adjust the content so that the average time of study per unit complied 
with the University guido-lines, and to allow removal of or amendment to 
particularly difficult sections. 

The course team had expressed some anxiety about the early television 
progranrnes, so it was suggested that particular emphasis be put on the evaluation 
of the first five television programmes, in order to 

(iv) determine what proportion of students claim that a particular 
television programme was totally unhelpful (or worse). 

Resources allowed for the remake of the equivalent of one television programme, so 
that if the evaluation data so indicated, changes could be made to a progranvne 
causing particular problems. In addition, broadcast notes could be rewritten, 
if necessary, both to help iron out any difficulties in the associated programmes 
and to deal with problems caused by the notes themselves. 

Since each television programme had associated self-asbo^sment questions in 
the broadcast notes, time spent on this work might have added noticeably to the 
overall study time for any particular unit. It was therefore decided to 

(v) obtain a *normal* time for study associated with a particular 
television programme. 

Information gained here could be used to make adjustments to the contents of the 
broadcast notes. 

In the case of the eight radio programmes in the course, the same sorts of 
question were raised. Additional evaluation aims were therefore 

(vi) to determine what proportion of students claim that a particular 
radio progranine was totally unhelpful (or worse) 
(vii) to obtain a * normal* time for study associated with a particular radio 
programme. 

Broadcast notes could be altered and several radio programmes remade if this proved 
necessary. 

Turning now from the specific problem of making possible adjustments to an 
already made course, to the more general issue of the presentation of future courses, 
the course team was anxious to know how students had reacted to particular pro- 
duction, organisation and presentation modes used in M231. Five specific areas for 
investigation were identified: 

(l) Pacing mechanisms used by students . 

The television programmes for M231 had been scheduled for transmission 
on the assumption that students do not study the correspondence units 
in tho weeks officially suggested in the course calendar (Appendix A), 
but that they lag behind and pace themselves against the submission 
dates for tutor-marked assignments* Was this a valid assumption? 
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(2) Use of self-agse»sment queationa * 

The printed unit»^televi»ion and radio programneB all had associated 
self -assessment questions (SAQs) in the broadcast notes (an example 
of broadcast notes is in Appendix B): those in the unit texts were 
described by the course team as "absolutely trivial". Did students 
actually use all or any of these SAQs, and how helpful did they find 
them? 



(3) The television progr 

The eleven television progrannes were planned to concentrate on some 
of the more difficult items in the course. In general they discussed 
and illustrated the solution of particular mathematical problems and/ 
or dealt with the proof of theorems. Two quite different production 
techniques were used to cover these separate programme functions. The 
»»problem-solution" was dealt with in a conventional "pen and paper" 
nmnner by showing a course team member actually working through pro- 
blems step by step and giving some verbal* explanation of what was 
happening »t each stage of the solution. The "proofs", on the other 
hand^were demonstrated by a novel and rather complex technique, which 
we can call "siJ.ent animation". This involved presentation of <tie proof 
by means of the animated manipulation of the various component signs 
and symbols, so that particular parta of the theorem relevant to any 
specific stage of its proof were highlighted sequentially. Very little 
academic comnentary - indeed in some cases none at all - accompanied 
the visual presentation, the intention being that its absence would 
provoke students into thinking more carefully than they might otherwise 
do about the general strategy or the proofs. How did students react to 
these two very different television techniques? 

(^) The radio programmes * 

- These eight programmes were intended to be problem-oriented rather than 
discursive; that is, they were planned to help students over difficulties 
with tho mathematical problems in the units. The associated broadcast 
notes contained material which students were expected to follow while 
actually listening to the prograrane. Did students, in the first place, 
listen CO the radio programmes? If they did, how helpful did they find 
the progranroes? Did they have any difficulty in following the material 
in the bi*oadcast notes during the programmes? 

(5) The tutor-marked assignments . 

Each assignment (TMA) contained one non-assessed problem which studentb 
were told could well be an examination question (only longer). Did 
students in fact attempt these non-assessed questions or did they find 
them frighteningly difficult? 
Information on these five particular areas would assist in decision-making 
on the organisation and presentation of future courses, notably M332, in the 
Mathematics Faculty, and should also provide insights into some of the problems 
faced by Open University course designers in general. 
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Mothod 

It will bo cloar from tliG preceding outline that this p<».fticular study 
presented a number of problems in terms of selection of appropriate data collection 
instruments and methods- Firstly, while much of the requested infornjation in- 
dicated the need for a year-long study (for example, to establish study times or 
difficulties of particular units of work), the course team needed some of the 
otherdata (for instance, for the remake of programmes and for decision-taking on 
the third-level course planned for 1975) as early as May or Juno of 1974- Secondly, 
although to answer some of the questions (the issue of vhe television techniques, 
for example) a fairly detailed approach seemed appropriate, many other aspects 
(usacje of various components, study pottern) could be adequately covered by a 
soriea of pro-coded options to questions- Thirdly, the number of students re- 
gistered for M231 - 769 at the beginning of the academic year - did not allow much 
leeway in the selection of tho necessary student samples - 

After discussion among tho evaluators and the course team, it was decided that 
use of several different data collection techniques was indicated and that tho 
evaluation should be carried out in stages to coincide with the course team»s 
deadlines for receipt of the various pieces of information- The methods used 
were as follows: 

(a) A random sample of 490 students registered for the course in February, 
1974, were each sent a pad of Course Unit Report Forms (CURF) at the 
beginning of the year- The form, although for data processing purposes 
necessarily designed along the standard lines used for all CURFs, covered 
the specific M231 problem areas which could be tackled by closed-option 
questions- (Appendix C) - Students were asked, in a letter - signed by 
the Course Team Chairman - accompanying the pad of forms to complete a 
form for each unit of work from Unit 3 onwards- (Spare CMA crlls had 
been used to ask questions on Units 1 and 2: this data is included in 
Appendix H) - Those report forms gave information on difficulties with 
tho written material, dates of study and hours spent on various course 
components, use of SAQs and reactions to television and radio programmes- 
While most of the answers were pre-coded, there was space for a small 
amount of open-ended comment- This has been typed up and is provided 
in Appendices J» K and M- 

At th* beginning of the year, I6 students wrote letters of refusal to 
take p^xrt in tho survey- Letters were sent to students who had agreed 
to participate, at the beginning of May and in mid-June, reminding them 
to return their completed i^eport forms- Even at the beginning of the 
year, the response rate was a disappointing 48^6, with 235 students 
returning usable forms for Unit 3> TV2 and Radio 2- Throughout the year, 
the rate of return deteriorated (Table l) so that only 21 persistent 
students returned the final report form, covering Unit 16 and TVll- 
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Tabl» It Studantg Returnino Courae Unit Report Foma 



Foni 


M#Hponw n V • 




Original 


Response 


Registerad 


Adjusted* 


nuMb#r 




students 


Miaple 


rate 


■tudants 


response 

rate 


1 


U3,TV2,B2 




490 


48% 


7t>9 




2 


U4,TV3 


219 


490 




707 




3 


U5»XV4,R3 


210 


490 


437* 


7t>3 




4 


U7,TV5,R4 


179 


490 


3«7* 


7t>3 




K 


U8 


148 


490 


30S 


746 


30* 


6 


U9,TV6,R5 


144 


490 


29% 


746 


30% 


7 


U10,TV7 


109 


490 


22% 


740 


23% 


8 


U11,TV8,r6 


97 


490 


20S 


740 


21% 


9 


Ul3tTV9,R7 


71 


490 




740 


15% 


10 


Ul4 


63 


490 


13% 


723 


IW 


11 


U15,TV10,R8 


33 


490 


7% 


723 


7* 


12 


Ul6,TVll 


21 


490 


4% 


661 


5% 



• The adjusted response rate assuiaes that students in the aample withdrew in a 
similar proportion to the total course population. 



Even with an adjustn^nt to allow for withdrawal frow the course 
throughout the year, the response rate ie poor. Of course, the number 
of students actually registered 'for a course ie not always an accurate 
reflection-of the number of students eeriously following that courses 
for instance, although 661 students were officially registei-ed for M231 
at the end of October, only 5^ actually sat the end of course exam- 
ination at the beginning of November, i.e. 1?% of registered students 
were not examined. However, further adjustment to allow for this makes 
very little difference - e.g. the finally adjusted response rate for 
Unit 16 is pushed up to 6% - because of the very small numbers returning 
forms at the end of the year. Howver we look at the data then, we are 
forced to conclude that it must be treated with great caution for the 
second part of the year in particular: while a calculation of two 
standard errors on a range of data from the first report form indicates 
sampling errors of between i 3% to i 5% et the 959^ level of confidence, 
the same calculation for the final CURF indicates an error range of 
- 13% to i 21% (i.e. if 50% of those returning the last report form said 
that they TVll, we can be 95?^ certain that between 29% end 71% of 
the total student population saw the programme). Clearly, this is such 
an * «normous margin of error that the data is virtually worthless. 
However, up to and including CURF number 8 (that covering Unit 11, TV8 
and Radio 6) maximum eampling errors are below - 10% . 
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(b) Five separate xundom samples of just under 6o students registered for 

the course in February, 197't wero each sent a more detailed questionnaire 
covering one of the first five television programmes nnd, in the case of 
four of the samples, one of the first four radio programmes in the course. 
Xo individual student, then, was asked to fill in more tluxn one of those 
more detailed questionnaires, nor *rt5re any of the students in these five 
samples included in the larger sample to whom CUKFs had been sent. Those 
questionnaires (referred to subsequently as "broadcast questionnaires" or 
"DQs'* - Appendix D) asked for students* detailed and specific reactions 
to the television and radio programmes and also enquired about dates of 
study of the various units. As a cross-check some of the questions 
asked were identical to those asked on the CURFs, but there was much more 
space for open-ended comment, and these arc provided for reference in 
Appendices L and N. 

Students who had riot returned their qu est iohna ires at intervals of ten 
days and twenty days after the last transmission of the relevant programme 
were sent letters reminding them to do so* The response rate here was 
good, ranging from 91% to 79% (Table 2) when adjusted for students who had 
withdrawn from the course by the time of the relevant broadcast. 



Table 2; Students Returnina Broadcast Questionnaires 













Students with- 


Adjusted 


Questionnaire 


Programmos 


Responding 


Total 


Response 


drawn at 


response 


number 


covered 


students 


sample 


rate 


broadcast time 


rate 


1 


TVl 

Hi 


49 


58 




2 


87% 


0 


TV2 
R2 


50 
51 


59 


85% 
86% 


1 


86% 
88% 


3 


TV3 
10 


49 


59 


83% 


5 


91% 


'* 




47 


59 


80% 


2 


82% 


5 


TV5 


45 


59 


76% 




79% 




R4 


46 


78% 


2 


81% 



BecauSi of the small numbers in each of these five samples (each sample 
representing about 8% of the total student population) the sample errors 
arc rather large (within the range - 8% to * 14% at the 95% confidence 
level). However, other studies in this series ^ have found that high 
response rates are necessary to ensure accurate measurement of viewing 
and listening figures: for example, there is a tendency for non-rcspondoi?ts 
to withdraw or to do less well ill the final course assessment than 



1. See for example. Bates, A.V. 1975i Broadcast Evaluation Report No. 3. T291 
"Instrumentation**. 
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•tttdMits who mspoiKl to qM^tionmiro ounr^yoj voroovor, studonts who 
Mtes^uontly withdmw «r« IDwIy to watoh tolovision and - 
partiouUrly - liston to radio broad^sts ^. Conaoqpiantly, a low ro- 
•ponao lata la likaly to giy a biaaaad f iotura of actual atudant 
bahaviouri avan thouQh| aa wa hava oaaiii tha thooratioal aaapla arror 
Miy ba ralativaly low. Ftor thla raaaon wa baliava that tha data froa 
tha 809( to 90% of atudanta ratumlng tho broadoaat q^aatiol«lairaa 
roflaota, with raaooiwbla aeouraey« tha aetual bahavloMr (in tanM of 
viavino and liataning) of tha toUl 1974 Wt atudant population, and 
of ita rvaotiona to tha talaviaian and mdio proqraMa in q^^'tion. 

(c) A randMi aai^la of V) atudMta with talapKonoa, rafiatarad for M231 
in tha niddla of May 1974« waa aalactad ta ba intanriawad uaino « 
aohadula (Appandix B) which tMd baan pilotad at a Saturday School in 
Cai^ridga atudy oantra in tha pravioua Iterch. lha intarriawa waro 
intand»d to dalva aora daaply into raactiona ta tha Ma31 radio and 
talaviaion tachniqiuaa thait waa peaaibla through tha poaUl ^ationnairaa. 
A fUrthar ai« waa to aupply tha oouraa taa» with additional inforwation 
on tha uaa of SAQa in ordar to halp tah» a dMiaion dioUtad by a print 
daadlina for N332. In addition, doUilad ^ationa waro aakad about 
TV5, tha prograMM broadcaat innadiataly prior to tha talaphona in- 
tanriawat an analysia of atudanta* anawara ia praaantad aaparataly, in 
Appandix F. 

Of tha M«vl«d atudanta, 3 thay had withdrawn or wcpo about to 

do ao and 2 could not apart tho tiao to ba intanriawad. A fUrthor 10 
- daapita paraiatant afforta - could not ba oonUctad <htrino tha intar- 
viaw pariod aiat M»y to 2nd Jum. Intarviawa wara thua obtaiiiad with 
25 of tha original oaapla. 



mMVVtS 

WMMC PATTgHH AWP STVm TlWtS 

InfonMtion collactod in tha at^dy miOQ^at* that tho M231 oouraa taaa waa 
oorract in aaauaino that atudanta "lag baMnd« oouraa calandar aUrting dataa. 
(aaa p. a )• 

j^f Ifo ooneantrata hara on teU froa tha Couraa Unit Roport fonm for Unita 3-7 

rapraaantad graphically in Figura 1 (ovarlaaf). Bacauaa of tha lovar ralUbility 
of tha teta rotumad for latar unita in tha couraa (oaa p. 5 ) thia inforaation 
haa baan drawn on only to provida oocaaional coiq^rioon with tha pattara indioatad 
^ tli» dftta for aarliar work unita. 

luppoaa wa consider varioua hypothaaaa about tha way atudanta acha^la thair 
work. Otm hypothaaia night ba that atudanta work roughly to tha aohaAila of 



a. Qallaghor, M., 1975t Broadoaat Ivaluation Mport No. 4. B2kn\ >lnAiatrial 
Oiattiatry OMiponontn. 
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courso calendar start dates suggested by tlie course team. However, Fiyure 2 
(p. 9 ) shows that already by the beginning of Unit 3 there was a mean starting 
delay (from tlie scliedulod start date) of days. The mean delay increased 
dramatically to 26 days for Unit - possibly exacerbated by the exceptional 
difficulty students liad with Unit 3 (see p. 12). Dy the beginning of Unit 5 
mean dolay was 28 days - students were starting on average 4 weeks after tlie 
schodulod start date. For Unit 7 the mean starting delay had been reduced to 
21 days; students itviy liave boon helped by the "break" between Units 5 and 7 
(there was no text for Unit 6 so that students would liave time for catching 
up, consolidation or further work.) Moreover, the fact that Unit 7 is the last 
Unit covered by TMA 02 is important (see p. 11). The conclusion must be that 
students do not schedule their work according to course calendar starting dates. 

A second h ypothesis miglit be that students work roughly in a pattern dictatjed 
by the television bx-oadcast schedule. Since at least twice as many students re- 
ported watchimj the first transmission -Monday at 6.15 P»m» -of each programme, 
than the repeat-Saturday at 9.^5 rt.m. - we should take the first transmission 
date of any programme as the scheduling point. In fact, the majority of students 
had started work on the Unit related to each programme by the time of its first 
transmission (Table 3) 

TABLE 3; Percentage of students working on Unit related to TV 



First transmi 
Related to: 


ssion; TV2 
Unit 3 


TV3 
Unit k 


TV4 
Unit 5 


TV5 
Unit 7 


Students 
Working : 
On Unit 


6U% 


50% 


72% 


70% 



!!owever, it seems unlikely that the television transmission date i£, in fact, a 
crucial scheduling point for students. In the first place, the television pro- 
grammes were actually scliednled on tlie assumption that students would work behind 
the recommended starting dates for units; that is the first broadcast of any 
television prograrmic took place between 1 and 4 weeks after the recommended start 
date for study of its related^correspondence text. It was therefore to be expected 
tluit a respectable number of students should have reached the relevant unit by the 
time of the first transmission of each programme: it is perhaps a cause for concern 
that up to 50% had not done so. (e.g. in the case of TV3 - oven though this pro- 
(frantiiKS was transmitted 14 days after the scheduled start date for Unit 4 and even 
Ihouoh of students had finished work on Unit 3 by that time, see Figure l). 
An even "lower" point was reached for Unit <> and its associated programme, TV(>: 
at the time of the first transmission of TV6, only 38% of students had started work 
on the related Unit 9. In each case (TV3 and TV6) the programmes coinciile with the 



15 



11 



cut-off dates for TMAs covering previous units; it appears that these, rather 
than TV broadcasts, are the really inportant dates for students. 

This lead* to the third hypothesis: that students work to IMA due dates, 
or cut-off dtetes. The first tutor-narked assignoent, TM4 01, covering units 1-3 
was due on fterch 19th, and the cut-off date was April 5th (scheduled start for 
Unit 4 waa Iterch 9th). THA 02, covering units 4-7 due on Nay 21st, and the 
cut-off was June 6th (scheduled start for Unit 8 was May 11th). In each case , 
75% of students had finished work on the last unit for aach TMA (i.e. Units 3 
and 7) at the exact aid-point between due and cut-off dates (Figure l). In aach 
case, 75% of students had started work on the last unit covered by the TMA before 
the due-date. The iaportance of TMA dates in student's work iMttema is also 
indicated by the possibility that students (fhave a rest** after aubnitting an 
aasignvent: while 75% had finished work on Unit 3 by 27th March, it was l4th 
April by the tive Unit 4 had been started by 75% of students - .leaving a "gap»» 
of 18 dtiym (despite the fact that nost students were already well behind schedule, 
and that TV3 - relate-:!; to Unit 4 - was first transmitted on 26th March). This 
long gap between finishing work on one unit and starting work on another does not 
persist between Units 4 and 5 nor between Units 5 »nd 7- It does, however, reoccur 
between Units 7 and 8, suggesting that students nay well see the assigraoent sub- 
mission dates as peaks; after each ascent they may stop for a breather. 

Overall then, TMA due and cut-off dates are the best guide as to when the 
vast mjority of students will have looked at, or perhaps even studied, a particular 
unit. 

THE PRINTED UNITS AND TUTOR MARKED ASSIGNMENTS 
The Units in General 

On the whole, the units were found to be difficult (55% Baid »very»» or »»fairly" 
difficult over all units) and time-consuming (l4.4 hours is the median over all 
units). The study time is spread over ten days (from starting date to finishing 
date), and includes an average of almost four hours spent on assignments. 

Students were asked how long they spent on the Appendix (an optional item). 
Overall, 39% of students studied the Appendices, though this dropped to 31% iii the 
case of Unit 7, possibly because of pressure from the TMA 02 cut-off date. The 
median study time for the Appendix was one hour. 

At the beginning of the year, it was planned to make optional a number of 
other items - up to 3 per unit - these %*ere to be announced via Stop Press. In fact, 
only two of these ♦*stop press optionals" were announced, both in Unit 8. The fact 
that question 6 on the Course Unit Report Form asked about study times for possible 
optional material throtighout the year, makes interpretation of data for Unit 8 
highly dubious. The x'ery high number of ♦•no responses*^ may be accounted for by 
students who got used to ignoring thiu question: some of these non-respondents may 
in fact have studied the optional items. For Item A, 18 out of 45 respondents said 
they studied the section: the median study time was 1 hour. For Item B, only 6 
of the 45 respondents said they studied the section: 3 of these spent under 30 
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minutes on it, 1 spent between 30 and 60 minutes, 1 between 1 hour and ti hour 9^ and 
1 between li and 2 hours* 

Individual Units 

Results for individual units are given in Table 4 overleaf • Scores for Units 
13 onwards must be treated with caution due to the small number of respondents* 

Of the early units, number 3 can be seen to be extreme in several ways - e.g. 
most difficult and most time-consuming. 26% of students spent over 20 hours on 
this unit, and 9% spent over 30 hours on it. Generally, results seem to accord 
with expectations that limits and continuity (Unit 3) difficult concepts, 
differentiation (Unit 5) straighfo'rward, and Riemann integration (Unit 8) involves 
long-winded calculations. 

Difficult Sections 

In Unit 3 the overwhelming difficulty was with the concept of limits and the 
cpsilon/delta techniques. Further, more general, comments on the units indicate 
that students found Unit 3 something of a shock after the relatively easy first two 
units: in fact a number of students said that this was the most difficult unit met 
in any Open University course so far (MlOO, M201, M202, M251 were mentioned in 
comparison) . 

In Unit 4, a number of students had difficulty in applying the method of 
repeated bisection, but the majority had general difficulty in following the proofs, 
particularly in Section 5« There was also general difficulty with the problem 
sections. 

With Unit 5, areas emerging as problematic were Section 5 (proof of the Chain 
Rule) and Section 9 (directional derivatives), but general comments on the unit 
indicate that on the whole students found it interesting and enjoyable. 

In Unit 7, sections 4 (Darboux*s Theorem) and 8 (implicit functions) gave most 
trouble, but again on the whole this was one of the less difficult units in the 
course • 

Two specific points of difficulty were mentioned for Unit 8s firstly the 
partition arguments - in some cases just their application - were found difficult, 
and secondly the double integral %ras said to be confusing. 

In Unit 9 the most frequently specified difficulty was that of grasping the 
meaning of the characteristic function. A few students felt that there was in- 
sufficient explanation and not enough examples of arc lengths. 

Logarithmic differentiation and the domains of inverse functions gave trouble 
in Unit 10. Generally, however, students were fairly happy with this particular 
unit. 

There was general approbation for the opportunity for both revision and 
practice afforded by Unit 11 - found by many the easiest unit of the course. A 
general comment was the sense of relief and encouragement gained from being able to apply 
well-learned techniques, and many found the unit enjoyable after the 'abstractness « 
of previous material. 
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Comments on the other units are not dealt with herei because of the low 
number^ resjponding students* All coonents are, however, available in Appendix J • 

Self Assessment <^stions and Problems 

Overall, k5% of students found the SAQs very or fairly difficult. Difficulty 
of SAQs relates positively to difficulty of unit (e.g. of the early units, SAQs 
for Unit 3 were found most difficult, and those for Unit 5 least difficult - Table 
4). The vast majority of students found the SAQs very or fairly useful (859t). 
Usefulness of SAQs was also related to their difficulty: easier questions tended 
to be rated more useful, (see p. 32 for further correlation analyses}* 

Students* comments indicate that these SAQs were indeed a success - boosting 
morale which had been lowered by the more difficult problems* Altogether, the 
Impression gained from the comments is that students found the problems botYi too 
many and extremely difficult - therefore time-consuming* On the other hand, 
students seemed to lack the confidence to "select**, and were uneasy about skipping 
any problems, despite the optional nature of some of the problem sections* 

Tutor-Marked Assignments 

The course team had included one non-assessed question in each of the four 
tutor-marked assignments* Such questions were rather more difficult than the other 
assignment problems and were intended to prepare students for the sort of questions 
which they would meet in the final examination. However, as Table 3 shows, only 
abotit half of those returning the assignments attempted the non-assessed questions, 
a disappointing proportion when conpared with the number of students tackling such 
questions in assignments for MlOO, N201 or MST281* 



Table 5s Tutor Marked Assignments for M231 



TMA 


No. students 

submitting 
assignment 


No* attempting 
non-assessed 
question 


% of total 
attempting 
non-assessed 
question 


Mean 
assignment 
score 


1 


699 


409 


59% 


7-1 


2 


522 


245 


4796 


8.1 


3 


378 


187 


4996 


7-0 


4 


364 


184 


5t% 


7.1 



Each question in *lic assignments covered the content of a particular work 
unit in the course. If we look back at the last horizontal column of Table 4 
(p-13) we sec that the units which caused most difficulty to students, had the 
lowest assignment question means: Units 3» 4, 9 and 14 had particularly low 
moans^. Each of these four units was covered by a different one of the four TMAs. 
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It is possible, then, thftt the difficulty which students had with these four 
units, the high study time spent on each - median times of between fourteen 
and a half and seventeen hours - both contributed to the extent to which the 
majority were workino behind sct^lule and decreased students' inclination to 
atieiq>t the non-assessed questions in the four assignments. 



THE TELEVISION PROGRA>f!ES 
Viewing Figures 

Both sources of information used in the study (CURFs and BQs) show 
a fairly high initial viewing audience, falling off quite rapidly after the first 
few prognumnes. The broadcast questionnaires provide data on the first five 
progranees (Table 6) 



Table 6: Students viewing first five television programmes (BQs) 





TVl 


TV2 


TV3 


TV4 


TV5 


Total respondents 


49 


50 


49 


47 


45 


Number viewing 


41 


39 


33 


30 


26 


% viewing 


84% 


78* 


67* 


66* 


58* 



The CURF figures are consistent with these, and show that the non-viewing trend 
suggested by the broadcast questionnaire data contirues throughout the year. 

Table 7t Students viewim last ten television programmes (CURFs) 





TV2 

X 


7 


TV4 


TV5 
pi 


TV6 


TY7 


TV8 


TV9 


TVIO 


TVll 


* Viewing 


75 


63 


59 


64 


60 


55 


40 


65 


81 


74 



Reversal of the trend between 3CV4 and TV5 may be due to the fact that students 
had begun to "catch up" to some extent by the time of transmission of TV5: by 
the time of its 2nd transmission, 75* were working on the related Unit 7 (■•• 
Figure l). Alternatively, this may be not so much a reversal as a re -establishment 
of the real trend. TV4 may be artificially low because of the fact that there was 
a three week delay between TV3 and TV4, instead of the previously established 2 
week delay. This nmy have caused a number of fftudents to miss the programme 
unexpectedly. 



3. Throughout the results, differences between BQ and CURF data are not signi- 
ficant and must be accounted for by the ejror margin in the sample sizes. 
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Students w«re asked. in the broadcast questionnaires, if there was any 
particular reason for not having watched the relevant television progran»e. 
For each of the progran-«s, .nost of the reasons (over thre-quarters) were 
concerned with work or family coH«itmonts and with inconvenient transmission 
times, k few students mentioned being behind schedule as a reason for not 
watching, and a few others said they had not found Open University maths 
programmes crucial to the courses they had previously taken. No-one gave 
dissatisfaction with earlier M231 programnes as a reason for not watching a 
later one. 

For all programmes the weekday evening trwismission was more popular 
than the repeat at the weekend. Of CURF respondents, for each TV programme 
at least twice as many watched on Monday evening (l8.15) as on Saturday «omino 
(09.45). Generally, less than a fifth of students watched both transmissions, 
with the exception of TV2, which was watched twice by a quarter of the students 
in the BQ sample and by 23% in the CURF sample. 



Usefulness of the Television P rogrammes 

As far as overall usefulness of the programmes is concerned, we find that, 
of those who watched^most students found most of the programmes helpful. Table 
7 shows the reactions of BQ respondents to the first five programmes. 

Table 8; Usefulness of first five tele vision programmes (Bqs) 



Numbers rating programme; 


TVl 1 


TV2 


TV3 


TV4 


TV5 


Very useful 


7 


2 


5 


5 


5 


Fairly useful 


21 


l8 


25 


14 


18 


Not very useful 


13 


1 


1 


10 


2 


Not useful at all 


0 


7 


1 


0 


0 


Don*t know 


0 


0 


1 


1 


1 


Total 


41 


39 


33 


30 


26 



Porcentaae results from CUKF data show that reactions of th^ two groups of re- 
spondents are broadly similar and that TV2 clearly stands out as being least 
useful. 
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T>bl«. 9-. U...fulne.. of l*«t ten televialon progr>iaw (C»RF.) 



p«rcentao« 


TV2 


TV3 


TV4 


TV5 


TV6 


TV7 


TVfl 1 


TV9 


TVIO 


TVll 


rating progranwat 


% 


% 




% 




ml 
% 


mi 

'% 








Vary uaaful 


8 


17 


22 


23 


27 


26 


35 


29 


16 


36 


Fairly uaaful 


30 


59 


54 


62 


48 


56 


49 


53 


64 


50 


Mot very useful 


35 


18 


19 


13 


14 


11 


8 


16 


20 




Not at all useful 


26 


k 


3 




5 


2 




16 


20 


7 


Don't know 


1 


1 


2 




1 




3 








to answar 




2 


1 


2 


5 


5 


5 


a 




7 



In order to .How w to pr.>be the r^Utive u-efUln... "rating." of the 
vriou. progran--., (e.g. to know whether . low rating w.. related to prograam 
content or etyle or perhap- to .ome other, external, factor) -tudent- - in 
both BQ. and CURFb - were ariced to .ay whjr they had found any particular 
programme "not useful". A glo.. of theee co—nt. for each progra-n- i- 
given below. A complete li-t of verbatim co«ent. i. provided in Appendkd- 
K and L« 
TVt 

The general theme of these reasons was that the prograwie was too general, 
contained nothing new. and cam. too late, i.e. when work on Unit 1 had already 
been completed. Several students commented that this introductory progr 
was interesting rather than useful. 



TV2 

Almost all the reasons students gave for not finding TV2 useful were 
related to the silent animations, in particular to the pace at which they 
developed, and the impression of njumpiness" or nfUshing" which the sequence 
of changes of lines in the proofs seems to have engendered. 

In this case, most of those who found the prograimn. not useful said that 
this was because they had not done the necessary pre-study of Unit 4. This 
agreoswith our earlier finding that students tended to get "bogged down" with 
Unit 3« 

TV4 

Proof of the chain rule gave trouble - some said it was too fast, 

others that not enough explanation was given of certain steps in the proof 
(e.g. the introduction of the function called ^ in the broadcast notes). 
There was some feeling that the programme stuck too closely to the methods and 
approaches to problems and theorems that they had already seen in the textbook. 
Again, there were several people who had not yet studied Unit 5* 



TV5 

Very few comments were received here: (this was in fact, found to be one 
of the most useful programmes in the course). The most co«iion reason given 
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for not finding it useful was that the prograiane followed the textbook too 
closely. Again, the scheduling factor was relevant: for a number of students 
the prograimie was screened too late in relation to their study of the relevant 
printed imterial. 

TV6 

Here again, those who did not find the programwe useful were almost all 
students who had fallen behind the reconwended work schedule and had not 
studied the relevant text befcre seeing the progrmiwne.^ 

m 

This progranwe was generally criticised as having moved through the 
material so quickly that it was. impossible to follow the details. Moreover, 
both transmissions were on Saturday mornings, a fortnight apart. This unusual 
transmission pattern led to difficulties for some students* of those who 
normally watchod both broadcasts, some inadvertently missed the first showing 
while others forgot to watch the repeat because of the long del*y* 

Very few comments were received on the rertaining television programmes: 
they are all included in Appendices K and L. 

Reactions To Animated Proofs 

An attempt was made in the broadcast questionnaires and telephone in- 
terviews to learn how students reacted to the presentation of proofs by means 
of the manipulation or animation of words and symbols, as distinct from the 
more usual graphical animation. 

The overall impression gained was that students were not violently against 
the technique as such (nor were they fervently in favour of it), but that a 
substantial number of them felt that those animations they had seen in TV 
programmes 1-5 were too fast. In particular, the speed of the animated proofs 
in TV2, together with the fact that all such proofs in TV2 were silent, was the 
rmin reason why a large number of students classed this programme as not useful. 
Whenanimated proofs were voice-accompanied, students seemed to be nuch happier 
with them. 

It is quite likely that the widespread unfavourable impression of TV2 
coloured students' responses to the question of animation technique as used 
in later progranmies (especially TV3), and this should be borne in mind in the 
interpretation of the results which follow. 

One of the major aims of the animation technique was to increase students' 
concentration during these animated sections of the programme and hence make 
them think harder about the theorems than they might otherwise have done.. 
Students were divided in the extent to whi<A they felt that the animations 
were successful in this respect. 
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T,^,, c of xsiimtior' <n ln..r^.i«i r^n«*ntr>tion « ^B^.) 



NuwbT rating 
animationa aat 

Vary miccaaaful 
Fairly aucceaaful 


TVl 


Tva 


TV3 


TV5 


8) 

31 

23) 


J 13 
la) 


) 13 
11) 


} lb 

13) 


Not very aucceaaful 
Not at all auccaaaful 

No answer 


2 


16) . 
} 26 

10) 
0 


0 


'J' 

1 


Total 


41 


39 


33 


26 



• No ■il>nt »ni«»tioivi were »»«d in TV4. 

It ^pp^r. that the .ni-tion. h.v. not b«.n entirely unmicce..ful 

in thi. .i«, thouBh it i. difficult to .ay ju.t how ■ucc.ful they h.v 

been in view of the b««— HX n-8*tiv. r-ctio„..to TV2 .nd th. po..ibl. 

..h*n»-ov.r" effect on XV3. Not., for ««pl., th»t th. .nl«tioM in TVl 

particuUrly, but .l.o in TV5, wr. r«t.d having b..n on th. whol. 

«,cce..ful in their aim of increa.ing concentration. 

The producer of the pr«gr«.-.,aware that .tud.nt reaction, to the 

animation technique might well be negative, wanted to know whether «.ch 
reaction might actually have interfered with th. l-rT»ing pr^c..., or 
whether the pr«,f. de«on.t«t.d through ani-ation, n.v.rth.1... r.«in.d 
in viewer.' n—ories. Student, were therefore a.k.d which ani-at.d 
proof., from a given li.t, (not in the order .hown here) they recalled 
.» having been in the programme on which they were reporting. They were 
..ked not to refer to the broadcaat note, wh.n an«r.rino the que.tion. 
The re.ult., a. shown in Table 11 overleaf, indicate that recall wa. 
generally good. Student, filled in their que.tionnaire. up to three 
week, after viewing the prx>graB««5. th. amount of error in allocating 
particular theorem, to particular progran»e. i. th.refore not great. 

The technique having been e.tabli.hed in the fir.t f.w program.., 
reapondent. to the final two br.,adca.t qu..tionnair.. w.re a.ked how u..ful, 
in general, they found thi. «»imated demonatration of proof.. 
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TVI 


TV2 


TV3 


TVU 


.TV5 


No* respondljag to this 
question 




36 


32 






f2 Is irratlonil (ws 

In TVI ) 


37 


6 


1 


3 


0 


f,g continuous 

f,g continuous 
(was in TV 2) 


1 
1 




2 


1 


1 


r,g continuous 

fog continuous 
(was in TV2) 


1 


28 


0 


1 


0 


fx is continuous 
(was in TV2) 




25 


1 


0 


1 


Tntemediate value Theorem 
(Theorem 7-1 ) 
(was in TV3) 




1 


2o 


1 


4 

1 


Chain Rule (was in TVU) 


0 


0 


1 


27 


1 


Leibnitz' forimila (was in TVU) 


0 


0 


1 


2h 


'2 


Rollers Theorem (was in TV5) 


0 


0 


0 


0 


2U 


Mean Value Theorem 
(was in TV5) 


1 


1 


3 


2 


2U 


/a+b/i:/a/+A/ 
(Not proved in any prograrane) 


1 


2 


0 


1 


1 


X continuous, 

Ca^^l ^ bounded on a,b 
(Mentioned, but not proved, in 


1 


1 


15 


0 


0 


f,g continuous f + g contlmm 
(Stated, but not rroved, in TV2) 


0 


17 


2 


1 


0 


V (x)> 0 for IE (a,b) 

f increasing on (a,b) 

(Proved in TV5, but not using 
animation technique) 


0 


0 


k 


0 


15 


f,g dLfferentiable 

(fg)'« fg' ^ 2 (^s^d but " 
not proved) 


0 


0 


1 


7 


1 
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WUrtbT» rating animtiona ast 

Vary uaaful 
r^irly uaalUl 

Not vary uaaful 
Not u«a^il at all 

Don*t know 
No answar 



Total 



TV4 



7 
8 

10 
4 

0 
1 



30 



TV5 



5 

9 

8 

a 

1 
1 



a6 



Both 



la) 

17) 



Raapondants vara claarly fairly avanly aplit in thair aatipataa of tha 
ganaral uaafUlnaaa of tha tachniqMa. 

A final quaation concarmd tha paca at which tha aniaatad proofa 
davalopad. 

Tabla 13 1 Ovarall Paca of tha Aniwat^d Proofa 



NUMbara findii^ tha aninationat 


TVl 


Tva 


TV3 


TV4 


TV5 


Altogathar too faat 
Too faat in placaa 


13) 


> 


'i " 

16) 


'J « 




About right 


a4 


3 


10 


0 


14 


Too alow in placaa 


2 


0 


3 


4 


1 


Altogathar too alow 
No anawar 


0 

1 


0 

1 


0 

a 


1 
1 


1 
0 


ToUl 


42 


39 


36 


33 


26 



Nota that avan in TV 5f whare tha faw animtad proofa uaad in tha progi 
vara accoi^iad by grapha aa wall aa by voica cowantary, atill al«o»t half of 
tha atudcnta would hava likad thinga alowad down, wh?,la tha proportion holding 
thia viaw ia avan highar for tha othar prognuMaa (axcapt TV !)• 

On balanca, it appaara that tha animation tachnlqua waa not co«platcly 
rajactad by viawara, though undoubtadly tha paca at which tha animtiona davalopad 
in TV2 cauaad grava prt)bla«a - indaad problaM of a laaaar wagnituda wara cauaad 
by tha paca of tha animations in all progrannaa covarad by tha broadcaat quaation- 
nairaa. Thara ia nothing to auggaat that atudanta in ganaral wara particularly 
•for* or 'againat* thia tachniqua, although aowa atudanta dafinitaly did aaa« to 
dialika it. Alaoi it aaa»ad poaaibla that tha tachniqua aipht indaad hava 
incraaaad concantration; carUinly atudanta did not appaar to hava difficulty 
in racalling proefa da«>natratad in thia way. Howavar, in viaw of tha novalty 
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of the technique and the fact that students were "immersed" in it very 
early on in the course (in TV2 -afiirther complication being that this 
prooranwie dealt with the epiilorv/dclta techniques which students found 
particularly difficult in Unit 3) it is extremely difficult to distinguish 
between inherent disadvantages of the technique itself, and problems 
caused by the way in which it was introduced and applied in M231. 



Problems Worked Through By Presenter 

Students were specifically asked how useAil they found it to see 
problems being worked through by a presenter. The general impression 
gained from both questionnaire and interview responses was that this 
technique was certainly appreciated, primarily since this was the method 
roost likely to assist students in gaining some idea of how to go about 
"tackling" problems - the principal overall difficulty which students 
appeared to have with the course in general. 

Table l4; Usefulness of Problems Worked through by Presenter (BQs) 



Numbers rating the 


TV3 


TV4 


TV5 


All programmes 


technique as: 










Very useful 


16 


9 


. 13 


38 


Fairly useful 


15 


13 


' 11 


39 


Not very useful 


0 


5 


2 


7 


Hot useful at all 


0 


3 


0 


3 


Don't know 


1 


' 0 


0 


1 


No answer 


1 


0 


0 


1 


Total 


33 


30 


26 


89 



< Question not asked on BQs 1 and 2. 
Some students said that they preferred this kind of presentation to the 
use of animation because it resembled more closely a tutorial or class- 
room situation and was valued for the sense of personal contact provided. 
Several said that they were accustomed to this manner of presentation and 
therefore knew what to expect from it. 

Some specific problems worked through in TV3, TV4 and TV5, were 
mentioned >y students as being particularly helpful. 
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Tfcbla 15t Proble— H«lpfUl to Students In TV3-5 



TV3 


Helpful as 
concrete 
eaewple of 
Abstract 
ideas 


Helped to 

oo ovneA 
WO lie 


Helpful for 

Baviaion 


No. of tiaes 
■entioned 
as being 
helpful 


FroblaM 7-17 
p. 109 of Spivak 
iHotm 3 in broftd* 
cftst notes) 


4 


3 


1 


8 


Theore« 7-1 (Note 
4 in broadoaet 
notes) 


1 


2 


1 


4 


Shoe Fitting 
l.u.b. exaaple 


3 


1 


1 


5 



TV4 


Helpful as 

concrete 

exanple 


Helped to 
do other 
work 


Helpful 
for 

revision 


No. of tiaes 
Mentioned 
as being 
helpful 


£auuq;>le of 
Leibnitz fomula 
for 

derivative of • 
product 


4 


3 


2 


9 


Using iaplicit 

differentiation 

to find d;^ if 
dx 

X 4 y ai 1 and 
f^(x) if tan 
(f(x)>= X 


3 


1 


1 


5 


Pascals triangle 


0 


1 


3 


4 


Differentiation 
of Si«(2x 4 1)'' 


2 


2 


4 


8 



TV5 

Only a few students Mentioned anything specific here. The relation- 
ship, shown graphically, between Rolle's Theorem and the mean value 
theorea wa» pientiGned once, and the exai^jle of finding the derivative of 
arctan three tines. 
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Use of Broadcast..Jiotei» 

>iost .tudents (at least two-third, of BQ respondents and 80?i of 
CURF respondents for Prognu-*) »*id that they had, as intended 

by the course team, read the introduction to the TV broadcast notes 
before watching the progr«»«e. However, rather fewer students attempted 
the SAQs in the bn>«dc»t notes, especially those for the later progran»es. 

Table 16 1 U«ft of SAQs for first five television proqraimnes (BQs). 



^^mbera 
who: 


TVl 


TV2 


TV3 


TV4 


TV5 


Used 


29 


26 


17 


16 


9 


Didn't use 


10 


13 


16 


14 


17 


No amiwer 


2 


0 


0 


0 


0 


Total 


41 


39 


33 


30 


26 



Percentage results from CURFs show the saioe "dropping off" at TV5^ 
T.Mo I7t Use of SAQs for last ten television proqranwes (CURFs) 



Percentage 
who: 


TV2 
% 


TV3 
% 


TV4 
% 


TV5 
% 


TV6 
% 


TV7 

% 


TV8 
% 


TV9 


TVIO 
% 


TVll 

• 


Used 


71 


70 


73 


64 


74 


68 


81 


71 


48 


64 


Didn't use 


27 


25 


25 


34 


26 


30 


16 


29 


44 




1 Ko answer 


2 


5 


2 


2 




2 


3 


r 


8 





The "dip" for TV5 is difficult to explain, particularly as, like TV8 whose 
associated SAQs wero most widely used, TV5 was rated one of the most "useful" 
prooran»«s in the course. One possibility is that students were, at the 
time of the progninine's transmission, busy with preparation for their sub- 
mission of TMA 02. Some confirmation is provided by the other slight "drop" 
for TV7, transmitted near the due-date for TMA 03. Students were, in fact, 
asked wh, they did not use the SAQs for some of the programmes. 
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Tmhlm 18: R6*»on« for Not U»ing SAQp. (BQ») 



NuaikMrs giving 
following re««on«t 


TV3 


TV4 


TV5 


No tim 


11 


6 


11 


Vill use later 
(for revision) 


5 


2 


2 


Hot up to Appropriate 
unit 


2 


2 


2 


Other 


1(a) 


2(b) 


3(c) 



(a) Spent rest of evening after programs on MDT241, and forgot 
About SAQ's. 

(b) (i) Did not follow prograawe sufficiently well to answer questions 
(ii) Understood unit fairly well. 

(c) Two said »*lazy". 

One said -^because I had finished Unit 1 I considered this 

psrt of the course closed. 
Clearly, then, shortage of time was the major factor preventing 
students from using the SAQs. The students who Gsed these SAQs did 
generally appreciate their usefUlnesst between 80jt and 90* of those who 
used them found the questions "very" or "fairly" helpful. Those few 
students who gave reasons 'in the broadcast questionnaires) for i»t 
finding the SAQs helpful, tended to find it too easy to answer such questions 
iimnediately after the programme. 



Study Time Associated with Television P rogrammes 

In order to determine an Approximate nonsal time of study associated 
with each television programs, students were asked (in BQs and CURTs), how 
much time they had spent on the broadcast notes, including SAQs if used, 
after the television prograamie. 

Median study times have been calculated for those students spending 
at least some time on the broadcast notes after the programmes- 

Table Its Median time on broadcast notes after last ten f.levision programmes 



(CURFs) 





TV2 


TV3 


TV4 


TV5 


TV6 


TV7 


Ty8 


TV9 


TVIO 


TVll 


Median time 
(In minutes) 


26 


23 


26 


27 


30 


27 


38 


33 


24 


22 



30 
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The "high" for TV8 is probably caused by the high proportion of students 
attempting the SAQs (see Table l?) and the fact that there were more 
questions associated with this programme than was usual. 

Adding to this median about 5/lO minutes for reading the introduction 
to the notes, we estimate the normal study time associated with the 
television programnes (excluding the actual viewing time) to be about 
30-35 minutes: in other words, each television prograimiies in M231 
represented about one hour's work for the student* 

It is interesting to note that there is a consistent positive re- 
lationship between the time spent on the broadcast notes after the prograraroe 
and the usefulness rating given to the progranne by the student. On the 
evidence of the BQ*s, we found that for each of the 5 programmes, the 
average titie spent on the broadcast notes was higher for those students 
who found the programme very useful than for those who did not find it 
useful. 

This suggests that the SAQs are indeed having the desired effect of 
reinforcing the main points of the programme for those students who spend 
more time on them. Support for this comes from the CURF data (p33 ) where 
a significant positive relationship was found between helpfulness of SAQs 
and usefulness of the programme. 
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Th» Radio Progrj 



Listeninff rigurw 

The .tudy found a disturbingly low proportion of .tudent. li.t.ning to the M231 
radio progra-ie., even in the eerly part of the year. The broadcaat questionnaire, 
provide information on the fir.t four prograwiea in the cour.e. 



Tabl 



a 20; Students Listening to Firat Four Radio Progra—— (BQe) 



Radio 1 



Radio 2 



Radio 3 



Radio 4 



Total 

Respondents 

Ntmber 
Listening 



49 



30 



51 



29 



49 



22 



46 



14 



% Listening 



6\% 



57% 



45* 



30!i 



The CURF d*t* confirma that only about three-fifth, of .tud«it. li.tened to even 
the fir.t programme and, although reflecting rather higher overall li.tening 
figure, than tho.e indicated by the BQ data .how that fewer and fewer .tudent. 
li.tened a. the year went on. (Figure, for Radio 7 and 8 are not reliable: see p. 5) 



T^bla 21: Students Listening to La.t Se^-^^ P^dir. Pronrs— if^s (CURFl) 


% listening 


Radio 2 
% 


Radio 3 
% 


Radio 4 
% 


Radio 5 
% 


Radio 6 


Radio 7 
% 


Radio 8 


60 


53 


55 


49 


43 


51 


47 



Thi. low listening population wa. confirmed by response, to the telephone 
interview., where it was found that 8 of the 25 students interviewed had listened 
to no M231 radio progran«es at all, and that a further 3 h«l heard only the first 
progran«.e. Only 6 of the 25 had heard (or at least put on tape) all four progranmes 
transmitted up to the time of the interviews. 

The interviewees who had listened to no programmes were asked why this was so, 
and were specifically asked for their opinion, of any other maths radio programmes 
they might have heard. They were also asked whether they had looked at the M231 
radio broadcast notes at all, to discover whether they could have known what kind 
of radio programmes they were missing. 



Mo.t of them had read the broadca.t note., a. a po..ible .ource of extra help 



4. The CURF data almo.t certainly presents an inflated picture of listening figures. 
O See p. 5. ^ 
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in understanding the unit or in doing TMA or CMA questions, so most of those who 
chose not to listen to any of the programmes were aware of their probl em-sol vipg 
orientation. 

Four of the eight said they had not generally found past maths programmes very 
helpful, although two others thought they were very good* Five of them said they 
preferred tq spend the available time studying the printed units* This vras, in 
fact, the overwhelming impression gained, both from specific questioning and from 
spontaneous comments made by the 2^ students interviewed - radio is regarded as the 
least essential component of the course, the component to be sacrificed first, the 
component students are least willing to make an effort to use. This seems to apply 
even to those students who find the radio programmes helpfu^ • 

Among those who did listen to the programmes, preference for one particular 
transmission slot was by no means as clear-cut as had been the case for the 
television programmes* 



Table 22; Use of Transmission Slots for First Four Radio Programmes (bQs ) 



lumbers Listening t 


Rl 


R2 


R3 


r4 


Wednesday 7*10 o.m*-^ only 


15 


15 


12 


5 


Saturday 7*20 a*m. only 


10 


11 


6 


> 4 


Both transmissions 


2 


3 


1 


3 


Taped only 


3 


0 


2 


2 


No answer 


0 


0 


1 


0 


Total 


30 


29 


22 


. 14 


Taped also 


4 


4 


5 


4 



The CURF data confirms this general pattern, showing the Wednesday evening 
transmission to be slightly more popular, just under a tenth of students - on average - 
listening to both transmissions* 



Table 23: Use of Transmission Slots for Last Seven Radio Programmes (CURF) 



Percentaae 
listening *, 


R2 


R3 


r4 


R5 


R6 


R7 


R8 


Wednesday 7.10 p.m* only 


45 


40 


52 


45 


47 


43 


33 


Saturday 7*20 a.n* only 


29 


33 


31 


25 


21 


23 


60 


Both transmissions 


9 


8 


5 


9 


13 


11 




Taped sound 


17 


19 


11 


22 


18 


20 


7 



Note, however, that in all cases, quite a substantial proportion of students 
listening did so on Saturday morning, even though the broadcast was at 7.20 a.m. 
(Compare this with the 9-45 a.m. Saturday viewing time). However, the early time 
of this broadcast was given fairly often as a reason for not listening at all. Also 
worthy of note is the one-fifth - on average - who recorded the radio programmes on tape* 
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Uaefulnes* of Hadio Programweg 



Those student* who listened to the radio programmes generally found them 
useful, arti said that the pace was about right. 

Table 2k: Usefulness of First Four Radio Programmes (BQg) 



Numbers finding 
programme: 

Very useful 

Fairly useful 

Not very useful 

Not useful at all 

Don't know 

No answer 



Rl 



5) 
) 26 
21) 

2 

0 

0 

2 



R2 



7) 
)26 

19 ^ 

3 
0 
0 
0 



R3 



6) 

)20 
Ik) 

2 

0 

0 

0 



R4 



5) 

)ii 
6) 

2 

0 

0 

1 



Total listening 



30 



29 



22 



14 



Table 25; Usefulness of Last Seven Radio Prograiwnes (CURFs) 



Percentage finding 
programme i 


R2 
% 


R3 
% 


R4 
% 


R5 
% 


r6 

% 


R7 
% 


R8 
% 


Very useful 
Fairly useful 


23) 

)S5% 
62) 


31) 

m% 

52) 


33) 

)9C* 
52) 


26) 

)72* 
46) 


42) 

)89S 
47) 


29) 
51) 


60) 

)1 

27) 


Not very useful 


11 


9 


4 


t? 


3 


14 




Not useful at all 


2 


8 


2 


8 






7 


Don't know 


1 


1 


2 


2 


3 






No answer 






2 


2 


5 


6 


7 



So no programme stands out as being particularly not useful, although Radio 5 slumps 
somewhat in relation to the others, for no reason apparent from the comments made by 
students. Of those students who didn't find the programmes useful, the most common 
reason given over all programmes was the pace at which they developed, although, in 
fact, the majority of all students said that this P«ce was about right. (Table 26). 
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Table 26= Pace of the First Four Radio Progrfcinaea (DQ>) ' 



Numbers finding 
the programme: 


Rt 


R2 


R3 


R4 


Much too fast 


0 


0 


1 


0 


Rather too fast 


4 


4 


4 


1 


About right 


22 


25 


16 


11 


Rather too slow 


3 


0 


1 


1 


Much too slow 


0 


0 


0 


0 


No answer 


1 


0 


0 


1 


Total listening 


30 


29 


22 


i4 



♦This question was not asked on CURFi 



Programme Format 

Students who were interviewed were asked for their opinion of the format of the 
radio programmes- Over half said that they preferred the problem-solving style to 
any other. Two students would like to have had the main points of the unit 
discussed as well *s hearing problems solved, 2 were specifically unhappy with 
problem-solving via radio, and 3 called for "motivating" programmes and broadcasts 
which "linked up" the theorems in the course, in order to give a better idea of its 
general structure. One of these also liked MlOO historical programmes, and would 
like to have heard something along those lines. Nevertheless, all but the two who 
had no clear alternative style of programme in mind did find the programmes, as they 
were, useful. 



Use of Broadcast Notes 

All but a handful of those who listened to the radio programmes said that they 
read the introduction in the broadcast notes before listening, and the vast 
majority also attempted to follow the notes, as instructed, while listening to 
the programmes: some experienced difficulty in doing so (Tables 2? and 28). 



Table 2?; Difficulty in Following Notes While 
Listening to First Four Programmes (bQs) 



Numbers finding it: 


Rl 


R2 


R3 


R4 


Very difficult 


0 


0 


0 


0 


Fairly difficult 


2 


7 


5 


1 


Not very difficult 


17 


14 


10 


6 


Not difficult at all 


8 


7 


4 


3 


Didn't try to follow them 


1 


1 


3 


3 


No answer 


2 


0 


0 


1 


Total 


30 


29 


22 


l4 
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CimP data is corwistant with this gsneral picturet 



Tabls 28; Difficulty in rollowino Notss Whils 
Listening to Last Ssvn Prooraat— (CURTs) 



Psrcsntags finding it 


R2 




r4 










Vsry difficult 


7 


10 


4 


17 








Fairly difficult 


22 


18 


21 


23 


16 . 


.23 


7 


Not vsry difficult 


41 


42 


36 


34 


39 


46 


60 


Not difficult at all 


25 


23 


29 


23 


39 


29 


" 33 


Didn't try to follow thsn 


2 


4 


7 


2 








No answer 


2 


3 


2 


2 


5 


3 





Consistently, thsn, about a quarter of those who attempted to do so had sows 
difficulty in following the notes during the radio progrsMMS. There was also a 
positive relationship between usefulness of progras«es and ease of following notes 
during the«: note, for exaaqple, Radio 5. This was the prograwie rated least 
useful and the one during which students had Most difficulty in following the 
broadcast notes. Apart fro« the "dip" for Radio 5, there is aoM slight indication 
that during the year students "got used to" the activity of following the notes 
wMle listening, (table 28). 

About half of those who listened to the radio^ pi^ogra^es used the associated 
SAQs in the broadcast notes. All of those who used the« found the questions 
helpful. Reasons given for not using the SAQs were usually "lack of time", or 
"saving the« for revision", although one or two: said they didn't use thesi because 
they had not yet reached the relevant printed unit. 

Similar findings esierged from the telephone interviews, with slightly more than 
half the students questioned using the SAQs straight away. A couple of people did 
not like "filling in boxes", but one was very enthusiastic about this activity. 
Most thought it was a gooiJ idea to have SAQs in the broadcast notes. 

Study Time Aaaociated with Radio Programmes 

To calculate an approximate normal study time, students were asked how much 
. time they spent on the broadcast notes, including SAQs if used, after each programme. 
Median study timeu have been calculated for all those who spent anjr time on the 
notes after the programme. 



Table 29: Median time on Broadcast notes After 
First Four Radio Programmes (BQs) 





Rl 


R2 


R3 


R4 


Median time 
(in minutes) 


29 


18 


26 


30 
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Ta blft 30t Median time on Brondcaat No tes After 
Last Seven Radio Proc|rammeg (CURFs) 





R2 


R3 


R4 


R5 


R6 



R7 


R8 


Median time 
iin minutes) 


21 


27 


25 


26 


30 


35 


30 



Addinrj on 5-10 minutes for reading the introduction to the notes, we estimate the 
normal Rtudy time associated with the radio programmes (excluding listening time) 
to be similar to that found for the television programmes - about 30-35 minutes. 
So each ra-dio programme in M231 represented just under an hours work for the student. 

The relatively low median study time for Radio 2 (over a quarter spent less 
than 15 minutes on the notes) can possibly be explained by considering the nature 
of the notes and SAQs for that programme, compared to those for the other programmes 
in the course. For Radio 2, there were no ''boxes" to be filled in, and the SAQs 
were all of a general nature, wlicreas for the other programmes, the notes were full 
of boxes, and the SAQs generally involved calculations rather than definitions and 
theorems, tt is possible, then, that students can more easily resist questions 
which may involve them in some thought before even the beginning of an answer can 
be written dowti. 



RELATIONSHIPS BETVEEN TifE VARIABLES STUDIED 

The information from the Course Unit Report Forms for all 12 units covered in 
the study, together with their associated television and radio programmes, was 
computer-analysed on the following 19 variables: 

* 1, Difficulty of units 

2. Difficulty of SAQs 

3, SAQs not used 
Usefulmjss of SAQs 

5, Study timo (excluding assignments) 

G. Timo spont on assignments 

7. Television programmes not watched 

8. Usefulness of television 

9. Introductory notes not read before TV viewing 
10. Helpfulness of SAQs for television programmes 
n. SAQs for television not used 

12. Time spent on television notes 

13. Television notes not used 

1^1. Radio programmes not listonod to 

15. Usefulness of radio 

16. Introductory notes not read before radio listening 

17. Difficulty following notes during radio programmes 

18. Motes not followed during radio programmes 

19. Timo spent on radio notes. 
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There •re throe iMin problem* in the interpretetion of the data obtained from 
this enalyflis:- 

(a) the email number of unite (12) covered in the etudy 

(b) the email numbere of etudente actually reporting on these units, particularly 
the later ones 

(c) the fact that there was some variation over the various units, in the students 
who did respond: not all units were reported on by the 'ame group of students, 
since some "skipped" reports on particular units and, of course, many students 
returning early reports did not return later ones* 

Nevertheless, we report on some parts of the analysis here^ since significant 
correlations were found, confirming findings from other parts of the study and since 
a number of interesting points are raised about the relative import ce of various 
aspects of the course components to students' overall study patterns* 

Coi I elation* between Unit Variablee (1 to 6) 

Only one correlation here was found to be statistically significant t difficulty 
of the unit was very highly related (r - .9^) to difficulty of the SAQs for the unit. 
This is hardly a surprising result, since the SAQs are based on the material 
contained in the unit. However, a number of other tendencies in relationphip are 
discernible, pin-pointing the difficulty of the unit the dominant variable which 
affects all the others. 

When the unit is difficult, its study time is increased and more students 
attempt the associated SAQs. Moreover, a longer time is spent on the assignment 
question. Where students do attempt the SAQs on easier units, there is a tendency 
(r s .(t2) for these SAQs to be rated more useful than those for difficult units. 
This may be because of the function which SAQs seem to fulfill for students 
(for example, providing encouragement and support - see p. 14 : students would 
be unlikely to rate difficult SAQs as "useful" in this sense)} or it may be that 
a favourable reaction to the easier units has been carried over in a similar 
reaction to the associated SAQs* 

Correlations between Television Variables (7 to 13) 

A number of significant correlations was found here. It appears that the 
dominant variable is the proportion watching television. When this proportion is 
low, those who do watch are more likely to study the broadcast notes, to attempt 
the SAQs for the programmu and to spend more time on the notes. It seems, then, 
that as television viewing "falls off" throughout the year, students who persist in 
watching are also persistently &ctive towards the television-related materials. 
Further, when the viewing proportion is low, viewers rate the television-associated 
SAQs as more helpful (r«.69) altHough this is almost certainly affected by the fact 
that the television programmes themselves are rated as more useful by "persistent" 
viewers. Moreover, such viewers are more likely to have done the preparatory 
.broadcast note) reading for the programme. 



6. Significance is attributed to correlations wherer«0.7 or more 
5. Complete correlation matrices are given in Appendix G. 
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Tho plcturo which omoroos, then, is of a hard core of talcviaion viewer*, who 
may be in some way particularly attracted to the television medium and who 
consequently enjoy and fini useful both the prograiwnes and the related activities. 
On the other hnnd, such students ma^ simply be conscientious in their overall 
approach to their studies, workinfl through all the materials in the recommended 
sequence and finding them all useful. 

Correlations between Unit and Television Variables (l- 6 and 7-13) 

In (jeneral, it appears that increased time spent on the text of a unit reduces 
^he amount of a television-related activity: units which have long study times 
for the text, and units which have difficult SAQs are such thai fewer students 
study the television notes (r-.65 and « -.63 respectively) and those who do atudy them 
spend loss time on the notes (r-.63). On the other hand, units which are more difficult 
have more students watching the associated television programme (r«.72). This 
seems to indicate that although particularly difficult units may influence more 
students to watch related television programmes, in such cases students have less 
time to spare for all of the activities related to the prograimne itself. It looks 
ns if some students in these circumstances turn to the television programme almost 
in an attempt just to gain some extra insights or clues into eventual mastery of 
some particular problem in the printed unit material, rather than with the aim 
of understanding the full scope ani purpose of the television programme in itself. 

Correlation between Radio Variables (l4 to 19 ) 

Only two significant correlations were found here. Vhere the proportion 
listening to radio is low, more time is spent studying the broadcast notes (r-.80). 
Where students read the notes before listening, they find the notes easy to follow 
during the progronmc ami the programme itself useful (r* -.95). This corresponds 
with the finding for television. 

Correlation between Radio and Television Variables (? to 13 and l4 to 19) 

Where the listening proportion is low, the proportion viewing is also low (r« .66) 
However, those who do watch and listen more persistently find the television 
programmes useful .78) and the associated SAQs helpful (r= .86) 
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COWCLUSIONS AND RgCO>fgNDATIONS 

Th« M231 evaluation .tudy met out to anwer a nu-ber of specific question, 
raised by the course team about the organisation and presentation of material 
in the "Analysis" course. The data has enabled us to provide at least partial 
answers to these questions, and to indicate soim possible courses of action 
to the course tea- in relation to the W31 course itself •nd to the proposed 
third-level course 'K:oiiv1« Analysis". Many of these suggestions were made 
in an earlier, interim, report to the course team and, where feasible, have 
already been taken up. We list here the specific recommendations to the 
course team in relation to the two sets of original questions raised: while 
the first set of findings and reco-wndations (A, below) is rather specific 
in its application to the M231 course, the second to) is •ich more general and 
will be of interest to other course designers in^« University as a whole. 



SU>t<ARISED rlHDiNUS AND SUGGESTED C OURSES or ACTION 

A. Questions Raised in Relation to Adj ustments to Content of M231 

1. Whst is the *normal< time for the period of study for a 
particular unit? 

Average study time was found to bo high - the median was l4.4 hours 
over all units. Study time on individual units ranged from a "^ian of 
13.3 hours on Unit 10 to 1? hours on Unit l4: indeed, 3lS of students spent 
more than 20 hours on this latter unit and lOjl spent over 30 hours on it. 
Another extremely time-consuming unit was number "3. with a median -tudy ti*me 
of 16.4 hours: in this case, 26* spent over 20 hours and 9* over 30 hours 
on the unit. In all'cases, study times include an average of 4 hours spent 
on assignment questions for the unit. 

2. Which sections in the unit are a bnormally difficult? 
Outstandingly difficult overall units were, again, 3» "Liwlta and 

Continuity" and 14 "Sequences and Series": almost a quarter of the students 
rated these as "very difficult". Other difficult units were 4 on »»Continuous 
Functions", 8 on "The Riemann Integral" and 9 on "Properties of the Integral". 

Vlthin specific units, particularly difficult sections - mentioned by 
more than 209( of students - were: 

Unit 3, sections 1 and 2: epsilon/delta techniques and the concept of 

limits! problems. 

Unit 4, section 5: overall difficulty with problemsi al»o general 

difficulty in following the proofs and diffi- 
culty in applying the method of repeated bisection. 
Unit 5, section 5: Proof of the Chain Rule(although this was 

rated one of the easiest overall units). 
Unit 14, section 6: the problems caused general troublei 

3. Vhat is the *non«al* time o€ •txxdY for those sections of the course 
which can be wade optionalf 
Overall, 39% of students studied the optional Appendices, though this 
dropped to 3lS for the Unit Appendix, possibly because of pressure from the 
Q TMA 02 cut-off dste. Median study time for the Appendix was 1 hour. 
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InterpretAtion of data for the other optional section* - in unit 8 - 
is difficult, because of the particularly low response to the relevant 
question. However, it appears that* iiO?^ studied the optional Item A, with 
a median study time of 1 hour. Only 13?^ of respondents reported studying 
optional Item B, with a median study time of 30 minutes, 

4. What proportion of students claim that a particular TV programme 
is totally unhelpful? J 

With one notable exception the TV programmes were found useful by those 
who watched (at least 75% of students who viewed said that any one programme 
was "very" or '»fairly"useful) • The exception was TV2, whick 62% of students 
said was not useful. Almost all the reasons given for its lack of useful- 
ness were related to the silent animations used in it to demonstrate proofs. 
Particular criticisms were made to the pace at which the animations developed, 
and the impression of ".jumpiness" or "flashing" given by the changing sequences 
of lines in the proofs as various parts of the theorem were highlighted. 

5. What is the 'normal' time for study associated with a particular 
TV programme? 

Median study time is 35 minutes (excluding viewing time). THi» is 
calculated by adding 10 minutes - for reading the introduction to the broad- 
cast notes •* to the med ian time spent on the notes (including SAQ's if used) 
after the prograinne. The median has been calculated only for those 75% 
of viewers who -spent any time at all on post -broadcast work. 

6. What proportion of students claim that a particular radio programme 
is totally unhelpful? 

No programme so jfar stands out as being particularly not useful, although 
Radio 5 was rated rather low, relative to the others: 25% found it "not 
very" or "riot' at all" ise'ful. Orerall| however, 85% of students "who Hastened found 
the programmes useful. Of those students who didn't find the programmes 
helpful the most common reason given was the pace at which they developed 
(although in fact the majority of jdl students said that the pace was about 
right). 

7. What is the 'normal' time of study associated with a particular 
radio ^programme? 

Median study time is 35 minutes (excluding listening time). This is 
calculated, as for the TV study time, for the 80% of listeners who spent 
any time at all on the post-broadcast work. 



41 



-37 



SuooMted Coura#« of Actio n Concerning Adiu.taent of Content of 82?! 
1. Units 3 and 14 caused -jor' difficultie. for aost .tudent. in 1974 and 
called for som remedial attention. Unit 3, particularly, co-ing .o early 
in the course, wa. a proble. .ince the inordinate length of .tudy ti«e it 
required certainly contributed to the .eriou. extent to which many .tudent. 
were behind «ihedule at almost the beginning of the year. Part of the 
prxjblem wa. that .tudent. were somewhat mislead by the comparatively ea.y 
first couple of unit. intA thinking that the course as a whole would be ea.y: 
Unit 3 thus came a. something of a rfjock. 

If con.idered inappropriate, or impos.ible, to alter either the approach 
taken in Unit 3 or the amount of material contained in the unit, .tudent. 
could be helped by «>me warning that thi. ha. been found • difficult part of the 
cour.e, and by an adju.tment of the study schedule to allow more time fbr 
work on this particular unit. They might also be advised to work through 
the first couple of units fairly quickly. A further pos.lble .ource of help 
could be the provision of extra tutorial help, either in normal class tutorials 
or at a Saturday School arranged during the period in which students should 
be working on Unit 3. 

2. Students were spending, on average, considerably more than the notional 
10/12 hours permit. Undoubtedly, an important contribution to the high study 
time was the number of "Problem" sections in each unit. Althou* the problems 
in the Optional Sections in each unit were labelled "routine problems" and 
"harder problems", no such distinction was made between any of the other 20 
or so problems set in each unit. Students would certainly be helped by the 
provision of more guidance as to how they should approach these problems: 
problems could possibly be graded into easy/difficult, short/long categories 
so that students could select more easily. 

3. The second television programme was a strong candidate for remake. The 
original programme's assumption that students would have covered the basic 
concept of limits and epsilon/delta techniques, was misplaced because of 
the extent to which students were working behind schedule and the difficulty 
caused by the text itself. A remade programme could discard this assumption 
and start from basic principles. Animations, if used in the programme, could 
be slowed down considerably, and students given some more "voice-over" 
guidance to the development of the proofs in the programme, as, for ex«»ple, 
in TV3 and TV5 which students found more helpful. 

4. While no radio programie really appears to need remaking, very much 
more effort could be put into initially motivating students to listen to 
the programmes. 



42 

i 



-38- 



B, QuQgtions raised in relation to (b) ; presentation of materials for M332 

1. Do students lag behind the officially suooft8te«3 study schedule and 
pace themselves against TMA due dates ? 

The data »uggef<ts strongly that TMA due and cut-off dates are the best 

guide to when the vast majority of students will have looked at, or 

perhaps oven studied, a particular unit^ Students appear to work neither 

to the schedule of course calendar start dates, nor to the TV schedule. 

There is also an indication that students may 

"have a rest" aft^r submitting an assignment: while there was a long gap 

between finishing work on Unit 3 (the* final unit covered by the first TH\) 

and startilr^^owork on Unit 4, this gap did not persist between Units 4 and 
I 

5, nor between Units 5 and 7: it did, however, reoccur between Units 7and 8, 
This suggests that students may well see the assignment submission dates 
*in peaks: after each ascent they may stop for a breather, if not from OU 
work altogether, at least from work on one particular course. It may be 
that students taking 2 or more half -credit courses prefer to orgarilse their 
work in 2-week "bursts" on each course consecutively, rather than study 
several sots of different course material concurrently, 

2. What do students think of the SAQ's in the correspondence text? 

All but a handful of students claimed to use these SAQ*s. Overall, 
45?o found them difficult, 37?^ about right, and 11% easy* Difficulty of the 
SAQ*s related positively to difficulty of the units (for example of the 
early units, SAQ's for Unit 3 - the roost difficult unit - were found most 
difficulty and those for Unit 5 - the easiest unit - least difficult). The 
vast majority (Q3%) of students found the SAQ's useful. The course team's 
original description of these SAQ's as "trivial" is somewhat belied by 
the finding that almost half the students rated them "difficult", and that 
so many found them usefuL Students' conments indicated that the SAQs were 
indood ,1 success - often boosting morale which had been lowered by the 
more diffi< -It problems. 

3- Mhni is the student reaction to the "silent" animations used in 
the TV programme.^? 

0\ Tall, students appeared not to be violently against the technique 
as such, nor were they fervently in favour of it. But a substantial number 
of students felt that those animations they had seen in the first five 
television progr«Mnmes were too fast. When animated proofs were accompanied 
by at least some voice commentary, students seemed to be much happier with 
U»enu However, this was a technique new to most students, and because 
of the extremely negative reaction to the second^ programme and the possible 
"spill-ovcr" from this to later programmes, it is difficult to distinguish 
tho extent to which the silent animation technique itself was unsuccessful 
from (.ho rather unfortunate way in which it was introduced to students. 

4. Do students use the SAQ's in the TV broadcast notes, and if so, how 
helpful are they? 

Overall, 720^ of students watching the programmes used the SAQ's, though 
ihH had dropp<>d to 66^ for TV5, perhaps because of pressure from the second 
TMA due*'date. Shortage of time was the major reason given for not attempting 
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the SAQ*8. 

Those students who did use the SAQ*s generally found thea helpful (at 
least overall). The few students who gave reasons for not finding them 
helpful said that it was too easy to answer questions straight after the 
prograMie. 

5* Vhat is the student reaction to watching someone work through a 
problem on TV? 

Overall, students certainly liked this, primarily as this method. was 
most likely to assist them in getting some idea of how to go about tackling 
problems, which seemed to be the main general difficulty for students of 
M23X. Thmn was also some feeling that this kind of presentation was better 
than animation techniques as it more closely reseoAiled a tutorial or 
classroom situation, and because proofs and problems were '^explained" 
rather than Just being "presented"* 

6. Do the students listen to the radio progtaiwes, and if they do, do 
, they find them helpful? Do they find the written material easy to 

follow durinji^ each broadcast? 

It looks as though a very substantial number of students do not listen 
to the radio progra m m e s* About 60% listened to the first programme, but by 
the end of the year only about two-fifths - at a generous estimate - of the 
students were listening. The overwhelming impression, from the interview 
and questionnaire data, was that radio is regarded as the most expendable 
component of the course, consequently the coisponent to be sacrificed first 
when under pressure of work, and the component students are least willing 
to make an effort to use. Other evaluation studies have suggested that 
maximising the useaade of radio is a problem not just for* the Maths Faculty. 

Undoubtedly, the majority of those who listened to M231 radio prograimMS 
found them useful: as many as 90% (for radio 4) and never less than 72% 
(for radio 5)* 96me of this enthusiasm may, of course, be peculiar to this 
highly selected group of students, who are likely to be considerably more 
. highly motivated than the average. However, the use of radio on M231 , being 
problem-oriented, was found to be a highly suitable one in view of the 
already mentioned difficulty students are having with the problfms. 

All but a very few students who listened used the broadcast notes during 
the radio programmes. For all programmes 23-30^ of students said they had 
some difficulty in following the notes. There vas also a positive relationship 
between usefulness of programmes and ease of following the notes during them. 



Gallagher, M (1975) Broadcast Evaluation Report no. 8: E221 "Caught in the Net". 

o 

See Gallagher, M (1975) Broadcast Evaluation Report no. 4: S24 "Industrial 
Chemistry Component" for further discussion of this point. 
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7, Dc students attempt the non«-as8es8ed problems in the TMA'a? 

Only about half the students attempted these questions for the four 
assignments. This is poor in comparison with the simileu: assignment 
questions on MlOO, M201 aiid MST 28l. This may be partly attributable to 
the fact that each TMA contained one question covering one of the four 
most difficult units in M231, The amount of time that students spent on 
these units, their difficulty and the fact that the mjority were working 
well behind schedule, may have induced students to avoid the non-assessed 
questions, which were also in the most difficult in each TMA. 

Pogsiblo Action Concerning Presentation of M332 

1, Evidence that students pace themselves against TMA due and cut-off dates 
raises the question of television scheduling: if the programmes are scheduled 
exclusively on the assumption that students "lag behind'!^ a substantial number 
of students will have read the relevant unit weeks before* This was, in 
fact, the case in M231* Particularly after the 4 week gap between programmes 
3 and 4, many students were well ahead of the material covered on TV (For 
example, 2^% of students had started work on Unit 5 by 2nd April - 20 days 
before the first transmission of the relevant TV progranwe 4). Indeed, this 
was one of the Host common reasons given by students for not finding the 
television programmes useful* 

One possible solution might be to move the first transmission forward 
(where necessary) to the week after the scheduled start date for the relevant 
unit (in some cases the first transmission of M231 programmes was two, 
three or even four weeks after the scheduled start date)*. Data from the study 
suggests that the top 25% of students were no more than one week behind 
schedule. If the repeat transmission is kept (as it generally was for 
M231) back to three weeks after the scheduled start date, then 70r^7'^ of 
students should have started work on the relevant unit: thus the majority 
of students would achieve maximum benefit from the broadcasts* A percentage 
of the remaining students who had not started work on the related 
unit at the time of. the broadcast might, if they watched, find the programme 
useful as a "pacer"* 

However, "staggering" repeats in this way presents enormous problems 
for the University as a whole, when one takes account of the fact that 
any one half -credit course shares its particular transmission slots with 
several other half-credit courses (since programmes for any one course are 
transmitted only every three or four weeks) , and that agreement would have 
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to be reached with &U of these course tea»s, if • »»staggered" transmission 

pattern was to be adopted. Moreover, if the availability of broadcast tim continues 

at its present level, and if courses are produced in their currently 

predicted nurt)ers, most courses will eventually lose their repeat transmissions 

«id will be left with the very much greater problem of how to maximise 

the benefit to students of broadcasting programmes once only. 

A somewhat simpler, and perhaps more obvious way of approaching the 
problem indicated by the M231 -tudy would be to move assignment due and 
cut-off dates closer to the reconmiended reading dates for relevant printed 
material. Of course, a fundamental and related question is simply the 
amount of work expected of students in order to complete both the Course 
as a whole and particular units within the course. If the course, or parts 
of it, are over-loaded, students will hare difficulty in even keeping pace 
with the assignment dates and will be forced to omit certain course components 
from their studies. 

A further problem raised by the relationship of assignment dates to 
broadcasiB centres on the issue of the range of functions which can be 
assigned to programmes transmitted at particular times - or at specific points 
in the »>assignment cycle»». If, for example, a programme is made dependent 
on a certain amount of pre-reading, its transmission date should be planned 
with the knowledge that not only will students need a certain minimum. amount 
of time for preparation, but, depending on the time- relationship between 
prograamie transmission and assignment due/cut-off date, that students 
at particular points of the study year will be more or less able an4/or 
willing to prepare themselves adequately. Consideration of factors such 
as these should, during the early stages of course planning, influence 
decisions as to what sorts of programme will be most appropriate at certain 
times of the year. 

2. It seems that, on balance, the silent animation technique need not 
be rejected. There is nothing to suggest that students in general were 
particularly "for" or "against" this technique, although some students 
definitely did dislike it. On the other hand, some student* were 
enthusiastically in favour of the method. Also^ it is possible that the 
technique, if used moderately, ija^ increase concentration, and certainly 
students did not appear to have difficulty in recalling proofs demonstrated 
in this way. However, in view of the novelty of the technique and the 
fact that students were "inmiersed" in it very early on in only the second 
programne in the M231 course (a further complication being that this 
programne dealt with the epsilon delta techniques which students found 
particularly difficult) , it is extr..u^ely difficult to distinguish between 
inherent disadvantages of the technique itself, and problems caused by the 
way in which it was introduced and applied. 
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It would seem altogether more reftsonable, both to the students, and 
to the reception 6f the technique, to .introduce now techniques such as that 
of silt-nt animation much noro gradually, building up from simple short 
animations with .some vocal guidance to longer, more complex ones later 
in the course- 

3, In view cf the large numbers of students who appear not to listen to 
any radio programmes, and who may not have done so since early in their 
Foundation year, very much more effort could be put into "pushing" the 
radio component at the beginning of the course- If the radio programmes 
are rcicogniscd by^ students as being closely related to the content and 
t;entral problems of the unit material, they will be much more likely to 
listen to at least the first programme on perhaps a "trial" basis- The 
oxtent to which tliey find the programmes helpful in getting to grips with 
this content and in solving these problems will determine to some degree 
thoir listening behaviour for the course- 

4- In view of the difficulty, which some students expressed, in concentrating 
on radio programmes, the use of notes such as those provided in M2'31 seems 

a useful way of not only holding students* attention while they listen, but 

of giving a visual anchor to the programme content after the broadcast- 

The SAQ»s in the notes, which students found" helpful , are an extra source 

of content reinforcement- However, the format of the M231 notes might be examined 

for possible areas of improvement since a large minority of students did 

have trouble in following them during the programraes- 

5- The extent to which non-assessed problems in TMA's are attempted will 
dopcnd on a number of factors such as the overall work-load and difficulty 
of the course, difficulty of assessed TMA questions and the amount of time 
involved in answering these- However, the difficulty of these non-assessed 
problems is in itself almost certainly an inhibiting factor: if they looked 
oasier, mor«? students would probably attempt them- Again, perhaps a grading system 
would help, .so that even the less able or motivated student could attempt 

at l<>ast part of the question* 
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